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(54) A method and apparatus for suppressing blocking artifacts in block-transform coded 
images 



(57) An improved method for removing blocking arti- 
facts in regions of slowly varying intensity in an elec- 
tronic image decoded from a transform coded 
representation of an image, includes the steps of: 
receiving signals indicative of DC and AC coefficient 
values of transformed blocks of image data: modifying 
the DC value in each of the blocks based on the level of 
activity within the block and the DC values in neighbor- 



ing blocks; performing AC prediction in each block using 
the modified DC values; replacing the AC values in each 
block with the AC predicted values based on the level of 
activity within the block and the modified DC values in 
neighboring blocks; and reconstructing the image using 
the modified DC and AC coefficient values. 
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Field of Invent inn 
Background Of The lnvflny /.« 

pressed f. e.. rKonsiructMl. A " 1 .™.r°T"" ««><KstatK». whe,e the image dete is decom- 

lossy JPEG (Joint Photographic Expert Group) international standard ' ^ transform (OCT), such as 

. ^o^Z:SZ'^:ZXX::Z:^ ^ '^on-o.^.^n,. contiguous 8x8 

representation space into a OCT J^Za^^ZT^J^^n!i^ J^^'l OCT, from a pixel 

coefficient and 63 AC coefficients ThToTcSe^^^^^^^^^ ''^ ~-P"^ of one DC 

ficlents represent the spatial frequency inforTaSn the btock °* ^'^ coef- 

. P^ior^nZSio^^ 

ping a range of coefficient values to o?eTa?ue Z^^m^n^r^i I^!^^"^' map- 
ing assigns a binary code to the rStlia ^7^^^^^^ 

decoded and dequantized to recons?^^^^^^^ receiving workstation, the bit-stream is 

subsequently trrnsformed fhe n.^e, r^Sse^S .^..T^'^^^^ ^^^^<^'-^ coefficients are 

is well known in the art. ^ representation spdce via an inverse discrete cosine transform (IDCT). as 

thenltrXa'Si^S^^^^ 

ration, will generate a l^eTnu^Tjl^^^^^^^ "^""^ ^ °* quantization levels, coarse quanti- 

undesirableblocJg^Ssatte J^S^^^^^^ coarse quantization intr^uces 

DC coefficients of neight^tg bKlJS?t Z?.1 Z'ln T ' '"<=^ease the disparity between the 

Conv'e^rgeTeSTslXb^^^^^^^^^ 

is the case in regions oJ'sloJy Sryir^ i^S ' ''"^"'"'^ ^^""'^ ^"^ ^^e detail as 

and is applied at the decompre«to7waSon nnm pr..>inn '^t^nrlnrrt 1993, page 261-265. 

dequantized DC coefficients f romTe crent wSk .nS f.? m T ''"^""""^ coefficients is formed using 
values for the block replace the Ton^esS,nd^^^^^^^ "T'* ""^''^^ ^C predicted coefficient 

into the Pixel representation spaS S shttc^S^^^^^^^ *° transforming the image back 

dequantized AC coeffidem SS re^,^"^^^ P^«^'<=*«^ <=oe«icients replace the 

area or a "quiet" area Sai ?h^?i^r.Nl ? h ^""^ °' block is in a busy" 

"busy" areas in the wJd< and Jin^Lur^^^^^ tendency of visually smoothing out high frequency image detail or 

Of dequantized DC co^ci^^ Llferj;^^^ 

De Garid?r:r ^nXilVS^a^rgasl^^J^ ^" ''''''' °^55^3A2 by 

eight nearest rieighbor Wocte TfTsfeS^^^^^^^ "'S °* ''^"^ "ock and its 

information via AC predion ^LVv-t^^^^^^^ L above-descnbed method, does not introduce low frequency 

high activity blocks is o «; ';h thTc "r^^ ''^^^"^y ^ ^^^^ '"fe^-^ation which is indicative of 

™rei:L';e' c-S^r"*"'"^ " "^-^ <^ l-a,es so as „ 
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Summary Of The invAn^ nr. 



accoZ'ronyr;iS;MSeX^^^^ -"^J °' '^'th above. Briefly summanzed. 

Slowly v^ying iritensrt^nl JeSroni^iZ"^^^^^^ "^'T^^ ^^'^^^'■"^ "locking artifacts in regions of 

-stepsof:recei..Uirr;SzrD?rAcr^^^^^^^^^ 
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dequantizing the quantized coefficient values; modifying the DC value in each of the blocks based on the level of activity 
within the block and the DC values In neighboring blocks: replacing the AC values in each block with the AC predicted 
values based on the level of activity within the block and the modified DC values in neighboring blocks; and reconstruct- 
ing the image using the modified DC and AC coefficient values. 

It is an object of the present invention to overcome the above-described shortcomings. 

It is another object of the present invention to apply the invention without requiring a change to the standard nature 
of a JPEG compressed bit-stream. 

It is a feature of the present invention to provide modifying the DC value in each of the blocks based on the level of 
activity Within the block and the DC values in neighboring blocks. 

It is an advantage of the present invention to provide a method which may either be implemented via software or 
hardware for performing the invention. 

These and other aspects, objects, features and advantages of the present invention will be more clearly understood 
and appreciated from a review of the following detailed description of the preferred embodiments and appended claims 
and by reference to the accompanying drawings. 

BRIEF DESCRIPTIQM OF THF np^yyi^^*^ 

Fig. 1 is a schematic diagram of a computer system for implementing the present invention; 
Fig. 2 is a block diagram of the decompression system according to the current invention utilizing an improved block 
so artifact reduction technique: 

Fig. 3 is a block diagram of an alternative decompression system according to the present invention utilizing an 
improved block artifact reduction technique; and 

Fig. 4 is a detailed block diagram of the method used to modify the DC coefficients. 
25 DETAILED DESCRIPTION OF THE INVgMTinN 

In the following description, the present invention will be described in the preferred embodiment as a software pro- 
gram. Those skilled in the art will readily recognize that the equivalent of such software may also be constructed in 
hardware. 

^° -ur^nf^y^^ '° '® ^ ^^^'^"^ implementing the present invention. A digital camera 10, such as a 

• KODAK DC 50. includes a charge-coupled device, CCD (not shown), for receiving incident light through a lens 20 and 
fw converting the incident light into an electronic, digital representation of an image 25 that is contained within the inci- 
dent light, as IS well known in the photographic industry. The CCD transfers the electronic representation of the image 
to a personal computer card (PCC) 30 for storing the image for later retrieval. 

However, before storage of the image on the PCC 30, JPEG compression software which is stored on a memory 
module (not shown) of the camera 1 0 is used for compressing the digital representation of the image received from the 
CCD. Such compression is desirable for limiting the amount of memory necessary for storing the digital representation 
of the image on the PCC 30. 

The PCC 30 is inserted into a local computer or workstation 40 for permitting the representation of the image to be 
decompressed and viewed on a monitor 50 that is electrically connected to the local computer 40. Alternatively, the dig- 
ital representation of the image 25. which is still in compressed form, may be transmitted to another remote computer 
or workstation 60 where it is decompressed for viewing on a monitor 70. printing by a printer (not shown) or further 
processing by a remote user. For decompressing the digital representation of the image, a software program of the 
present invention is stored both on the local computer 40 and the remote computer 50 for decompressing the digital rep- 
resentation of the image while also reducing the occurrence of blocking artifacts for permitting such viewing. 

Referring now to Fig. 2. there is shown a graphical illustration of the software program of the present invention for 
reducing the occurrence of artifacts in the image when it is reconstructed. It is instructive to note that this software 
receives the image in a form which has been compressed by lossy JPEG compressor A standard JPEG-encoded. 
compressed bit-stream. E^s. is decoded via a decoder 80 to form signals representative of quantized DC and AC coef- 
50 f icient values of a plurality of transformed blocks of image data. 
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The decoder 80 also decodes and produces a quantization table Q^^ representing the quantization that was applied at 
the compressor workstation 60 to the DC and AC coefficient values of each block. The quantized coefficient values. 
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i anl'!t^^TnT^TT^ '° ''f * '"'''^'P*' ' ^ ' '"^^ ^ block where both 

and J take on numerical values ranging from 0 to 7 inclusively. Index values of i=0 and i=0 correspond to the DC coef 

STubtrr H~*H '"'"."'"^ '° ^« values" he'Soc Furth^^^^^^^^ 

the subscripts m and n denote the horizontal and vertical position of a block 90 from the plurality of blocks reiSy 

of th^ bSTr^speSeT^^^ '^^'"'^ '^"^^ "^'"'^''^^ v^ues'^^thinV blo^TarSTo^^^^^^ 

The plurality of dequantized transformed blocks of image data. 

tl!<^r-f^eSj^r'^VEnr^ *° ^^"'^^ ^'^P^'^ coefficients of neighboring 

eact bl^ ADAPTIVE DC CORRELATOR 100. described in detail below, modifies the reconstructed DC coefficient of 
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35 Diocks Of image data with modified DC coefficients. 
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The modified DC coefficients 
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\a fT^"!S "Ir " ADAPTIVE AC PREDICTOR 1 10 to form adaptive AC predictions (as is wdl known in the art) tor low 
50 frequency AC coefficients in the block. The equations describing the predicted AC coefficient values are: 
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t^oT] m„ = (0.27881 / 8)((ZoT] m - 1. n + l^OO^l n,.^.n■ 2IZ°°] 



The output of the ADAPTIVE AC PREDICTOR 1 10 is transformed tjlocks of image data with modified DC and AC coef- 
ficients. 
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The modified DC and AC coefficient values in each block 
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20 are made consistent by an AC/DC CONSISTENCY CHECK module 1 20. The AC/DC CONSISTENCY CHECK module 
120 with inputs 
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and the quantization table. Qtab produces a consistent set of DC and.AC coefficient values which are denoted as 
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The DC and AC coefficient values in each block 



35 



■J Jmn 



are consistent if. when quantized by the corresponding 0,3^, entry, they result in quantized DC and AC coefficient values 
40 which are the same as those in 
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is not consistent, the AC/DC CONSISTENCY CHECK module 120 increases or decreases the appropriate coefficient 
values in each block 
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until the corresponding DC and AC coefficient values are made consistent. This method of consistency checking intro- 
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duces the smallest magnitude change of the coefficient values. The consistent blocks of DOT representative data. 

are then transformed back into a pixel representation space by a well-known INVERSE OCT module 130 to form blocks 
of reconstructed image data. 
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An alternate preferred embodimettt of the present invention is shown in Figure 3. The modules in Figure 3 which 
IS are equivalently numbered to those modules in Figure 2 are functionally identical. The embodiment in Rgure 3 is com- 
prised of an additional DC CONSISTENCY CHECK module 105. and an AC CONSISTENCY CHECK module 1 15 but 
^KfoJ'^^cill'^® ^^"^ combined AC/DC CONSISTENCY CHECK module 120 (shown in Figure 2). The DC CONSIST- 
tNCY CHECK module 105 takes inputs 



20 



and Qtab and makes 
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30 consistent with 
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35 in the same manner described for the AC/DC CONSISTENCY CHECK module 120 (shown in Figure 2) to produce 
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The AC CONSISTENCY CHECK module 1 15 takes inputs 
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and Qtab and makes the AC coefficients (i.e.. 1^*0 or j*0) consistent with 
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in the same manner described for the AC/DC CONSISTENCY CHECK module 120 (shown in Figure 2) to produce 



L '■• Jmn 



^NSOOCIO: <EP 0801S06A2J. 



6 



10 



15 



20 



EP 0 801 506 



a4I 



. As is well known in the art. the reconstructed Image data may be used by any suitable computing station, such as 
workstations 40 or 60. to transform the image data into a hardcopy image that has reduced blocking artifacts, particu- 
larly in regions of slowly varying intensity 

A detailed block diagram of the ADAPTIVE DC CORRELATOR 100 Is shown in Fig. 4. Dequantized transformed 
blocks of image data. 
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are passed to the EXTRACT DC COEFFICIENT module 140 which extracts the DC coefficient from each block which 
is denoted as 
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The DC low pass filter (LPF) module 150 generates a low pass filtered version of the 

1^00 
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25 denoted as 
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by convolving 
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with a low pass filter denoted as [B]. 
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q»-1p=i.i 



In the preferred embodiment the low pass filter 150. [B], is a 3x3 element matrix with the following coefficients: 

1/16 1/16 1/16 
[B] = 1/16 8/16 1/16 ; p=-l,0,l and q=-l,0,l. (3) 
1/16 1/16 1/16 



(2) 



55 

The elements of [B] are referenced as Bp^ with subscripts p and q corresponding to the horizontal and vertical location 
of the matrix elements with top left element being identified with p=-l and q=:-1. 

An alternate embodiment of the low pass filtering operation denoted by equation (2) is given by equation (4). 
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LOCAL ORA0modul/v70 TWmJIte^7?^ «r„!!* ■* COMPUTE MAXIMUM 



For all integer values of u and v such that. 
•V^u^ 1,ancl-l 1. 
so The original dequantized DC values 



^max=MAX(|[zS,I„,^.n.v <^Oo]mn\) (6) 
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low pass filtered DC values 
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the activity measure a. and the maximum local gradient A^^x are inputs to an ADAPTIVE FILTER 
erate an adaptively filtered DC coefficient. = r lui cn 



module 180 to gen- 



m ■ 

/5 The ADAPTIVE FILTER module 180 adaptively combines the 

[zO ] and [zQ ]■ 
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coefficients to produce 
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according to the following logic: 
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is combined with the original dequantized DC coefficient 
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only ^max is less than Arhresh- This is a switching mechanism which allows the adaptive filterina to be invoked in imaop 
regions for slowly varying intensity The adaptively filtered DC coefficient ^ 



:C10: <£P 0801506A2_L> 



9 



EP 0 119506 A2 



5 

replaces 



w 




by the INSERT DC COEFFICIENT 
modified OC coefficient, 

IS 
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As shown in Figure 2, 



25 




is an input to the ADAPTIVE AC PREDICTOR module 110 as previously described; and as shown in Figure 3. 
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Is an input to the DC CONSISTENCY CHECK module 105. 
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decoder 
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adaptive DC correlator 
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adaptive AC predictor module 




120 


consistency check module 
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inverse DCT module 
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extract DC coefficient 
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tow pass filter 
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compute block activity 
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compute maximum local grad module 
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adaptive filter module 
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insert DC coefficient module 



module 190 to produce a block of quantized transform representative data with a 
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l"c!^^°!i?m'^t»^i?' '^"^'^^ ""^^"^^ °' '"'^"=«y 3" electronic image 

decoded from a transform<:oded representation of an image, the method comprising the steps of: 

(a) receiving signals indicative of DC and AC coefficient values of transformed blocks of image data- 

(b) dequantizing the quantized coefficient values: 

(c) modifying the DC value in each of the blocks based on the level of activity within the block and the DC val- 
ues in neighboring blocks; 

Sr7Jn?^ tS ''•^"!! ^^^"^ ^^'"^ °" the modified DC values in the 
block and in neighbonng blocks; and 

(e) reconstructing the image using the modified DC and AC coefficient values. 

2. The method as in claim 1 , wherein step (c) further comprises computing the activity level by identifying a highest 
AC coefficient index that represents a non-zero frequency component in the block. 

^* IJcT?*^°1 as in Claim l ..wherein step (c) further comprises computing the activity level by calculating a maximum 
absolute value difference between the DC value of a block and its eight nearest neighbor^ for forming a l^^ra- 

Zn ^ J j? T ^•''!f '^^'"^ transformed blocks of image data and replacing the DC coeffi- 

rid cX^onTme? '^'^^ -«ge data with the convolved DC coefficient if a predeter- 

^' than a?r2how" ^' Predetermined criterion includes determining when the local gradient is less 

6. The method as in claim l further comprising performing adaptive AC prediction on the result of step (d). 

^' JUtlT^ ^IT^IS'" \ comprising performing a consistency check of the results of the modified DC val- 
ues ana au predicted values. 

^' fnr'lTl n?' ^^"^'^ ''^''^'^^ °' ^"^ Coefficient values of transformed blocks of image data and 

for removing artifacts of slowly varying intensity from the image data, the computer comprising: 

(a) means for dequantizing coefficient values; 

(b) means for modif ying the DC value in each of the blocks based on the level of activity within the block and 
the DC values in neighboring blocks; and 

Sh^'T^M ' ^^'^f '"^ ^^'"^^ ^^'^ P^«**««^ values based on the level of activity 

within the block and the modified DC values in neighboring blocks. 

^' Jof«t°ri!"*H as in claim 8 further comprising means for computing the activity level by identifying a highest AC 
coefficient index that represents a non-zero frequency component in the block. 

''^^'V comprising means for computing the activity level by calculating a maximum 

absolute value difference between the DC value of a block and its eight nearest neighbors for forming a local gra- 
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UnTgls"" ^"^^ ^P*^*^"*"^ ^""^ suppressing blocking artifacts in block-transform coded 



(57) An improved method for removing blocking arti- 
facts in regions of slowly varying intensity in an elec- 
tronic image decoded from a transform coded 
representation of an image, includes the steps of- 
receiving signals indicative of DC and AC coefficient 
values of transformed blocks of image data; modifying 
the DC value in each of the blocks based on the level of 
activity within the block and the DC values in neighbor- 



ing blocks: performing AC prediction in each block using 
the modified DC values; replacing the AC values in each 
block with the AC predicted values based on the level of 
activity within the block and the modified DC values in 
neighboring blocks; and reconstructing the image using 
the modified DC and AC coefficient values. 
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